Challenges and desiderata for a synthesis system

Here are some common failure modes for a synthesis system and process that I have experienced and observed in others (not mutually exclusive!):

Too much detail (too low-level, missing forest for trees). This manifests as a lack of higher-level synthesis of what a collection of results means. A common manifestation is the “x said this, y said this, z said this” form of literature review.
Too little detail (too high-level, missing the devil/diamonds in the details). This manifests as overgeneralization of claims, or glossing over critical inconsistencies or contradictions. A good example of this is debates about the role of “children” in COVID-19 transmission that ignore the details of differences between young children (under 10).
Insufficient context. This is related to the lack of details, but separate in that context can also come from connection to other claims: if this is missing, even observation notes can be lost because their significance isn’t recognized.
Information silos. This manifests in part also due to inordinate detail-orientedness, where important connections across disciplines or topics are ignored. This can also come from too little detail! If results are described at too high a level, we might miss important connections at the subproblem level between problems and results.
Information overload. There are often too many papers to read and process in a rigorous and iterative way, which leads to / exacerbates the preceding set of problems!